Detection of lost status of mobile devices

Users often forget or misplace mobile devices, e.g., smartphones. To find or recover such devices, users can utilize find my device features, if available on the device, cause the device to emit a sound, etc. However, no mechanisms exist for a device to automatically detect that it is lost or about to be lost. Per techniques described herein, an on-device predictor determines that a device is lost (separated from the owner) or is about to be lost. The prediction is based on various factors, including device sensor data, recent use of apps, recent user context, etc. for which the user has provided access. If it is determined that the device is lost, various mitigating actions as permitted by the user are performed. For example, such actions include sending notifications to other devices of the same user, initiating communicating with the user or a trusted contact, locking the device, encrypting user data, disabling notifications, etc

Smart assistance for meetings

The described techniques provide smart assistance to video conference meeting attendees. With user permission, such assistance includes replaying, transcribing, translating and/or summarizing all or part of a meeting, interpreting specified voice commands, and identifying topics or items of interest from the meeting recording or transcript. For example, meeting participants can provide keywords corresponding to topics of interest and seek notifications when such topics are discussed in the meeting. Also, users can query meeting recordings or transcripts for topics of interest or meeting action items. Present techniques can be utilized for video conferences, audio conferences, recorded talks, and face-to-face meetings

INTELLIGENT TELEPROMPTING SYSTEM FOR DELIVERING PRESENTATIONS

A system is proposed for intelligently and automatically advancing presentation text on a teleprompter User Interface (UI) as the presenter speaks. The system includes receiving, by a teleprompter system, a text file comprising speech of a presenter via the UI. The system further includes displaying contents of the text file on a UI screen. As the presenter speaks, the system further includes recording presenter speech according to predetermined criteria. The system also include running machine learning speech recognition tool to transcribe the recorded speech into recognized text. The system also includes semantically comparing text from the text file with the recognized text. If the recognized text matches content in the text file, the system includes advancing displayed text on the UI screen to show the following chunk of text to the presenter. On the other hand, if the recognized text does not match content in the text file, the system includes stopping the advancement of the text on the UI screen

Gaze and attention detection API for content pre-fetching

As software applications become increasingly sophisticated and complex, it is harder to ensure speed and responsiveness. For example, a sizeable fraction of content displayed by mobile applications is fetched in real time from the internet, because, e.g., it is not already available on the user device. To improve speed and responsiveness, applications attempt to anticipate user action and pre-fetch content. Techniques of this disclosure determine the screen area where a user is focusing by using gaze detection techniques, for users that provide permission for such detection. The gaze detection is performed using a front facing sensor of a user device and a model, e.g., a trained machine-learning model, or heuristics. With user permission, detected gaze data is provided to applications that can utilize such data to prefetch content, thereby optimizing usage of the data network and allowing for faster app interaction. Gaze detection can be implemented in an operating system, or within an application

Virtual assistant for planning and booking holiday trips

Booking and planning holidays is a time consuming activity that is performed multiple times by most users. This disclosure provides a virtual assistant that intelligently simplifies holiday planning and booking. A discovery and booking flow specifically designed for booking holidays is implemented to offer multiple options that the user can choose from at each step, or a limited set of complete plans are displayed based on user requirements determined based on user context, obtained with user permission

AUTOMATIC DUBBING OF VIDEOS WITH MULTIPLE SPEAKERS

A machine-learning model that automatically converts audio streams from an audio-visual content from a source language to a destination language is described. In response to determining that an audio stream should be translated, a machine-learning-based dubbing model is invoked for a specific destination language. In case of multiple speakers, voice embedding techniques are used to match dubbed audio streams to the corresponding speakers. The sentiment in the original speaker’s voice is preserved by training the model with targeted data set in the destination language

Dynamic context-based app permissions

Mobile applications obtain permissions from users to access device sensors and APIs. Once a user provides or denies permissions, the choice is permanent, unless the user chooses to change it manually and use of sensor data by the app is not within the user’s control. The techniques of this disclosure enable dynamic management of permissions provided to application software, e.g., mobile apps, based on user context. With user permission, a machine-learning model is trained based on contextual data and events, and corresponding app permission settings. The trained model provides predictions of when modifications to app permissions may be suitable. Based on the model predictions, users are notified to make such changes. If users permit, such permission changes are performed automatically when the prediction is associated with high confidence

API for learning and predicting user interactions

Software applications, e.g., mobile apps, at times require multiple user interactions to fulfill certain actions. This disclosure describes an API that returns a predicted user action, e.g., selection of an option in a software application, in response to a query by the application. The prediction of user action is generated using one or more trained machine learning models. The models are trained, with user permission and expressed consent, on prior user interactions with various apps. When users permit, other contextual factors, e.g., data from device sensors, other apps that are running, operating system data, etc. can also be used as inputs for the trained models. The requesting application can present the predicted action as a default setting, or can automatically use the prediction as the user selection. In this manner, the techniques can enable apps to reduce user interaction, e.g., the number of clicks/taps to complete an action

Determining the Relevancy of Permissions Requested by an Application Software

This publication describes methods that an application software market (application market) uses to determine a relevancy-permission score on an application software’s (application) permission requests to access resources, features, user data (e.g., calendar, photos, biometric data), and hardware (e.g., microphones, cameras, global navigation satellite system (GNSS), accelerometers). Depending on the request, an operating system (OS) or the application market may prompt a user to approve the requests. The user, however, may be unsure whether they need to accept or deny a particular permission request made by an application. The user may be unclear whether the access is needed by the application most of the time, some of the time, not needed at all, or whether the request is abusive. To aid the user make informed decisions, the application market uses fuzzing techniques, scripted journeys, data analytics, and machine-learned models to evaluate the permissions requested by the application. After such evaluation, the application market generates a relevancy-permission score for each permission requested by an application, which can be easily-understood by the user of the application before they grant or deny such permission requests

Screen brightness adjustment using machine learning

Mobile devices, e.g., smartphones, tablets, etc., often include a light sensor that senses ambient light. Data from this sensor is used to set the screen brightness. During ordinary usage of such devices, the light sensor sometimes gets covered, e.g., by the user\u27s fingers, which leads to a quick decrease in screen brightness. This makes for a poor user experience. This disclosure describes machine learning techniques that determine whether a detected change in ambient light is due to a true decrease in ambient light conditions or due to an occlusion of the light sensor. Screen brightness adjustments can be made on such determination